Allow serving static files from memory

[syeopite]

On high-load applications it'll be great if static files could be cached and served from memory rather than read from the disk each time. This will improve performance and prevent the application from opening a million file descriptors or from reaching the system's open file limit.

Over at the Invidious project we're using a patched version of Kemal's StaticFileHandler to accomplish this (not up to date with the latest version of Kemal):

https://github.com/iv-org/invidious/blob/5565204273de4140d7b72ab201adec1dd90ecf0c/src/ext/kemal_static_file_handler.cr

But it'll be nice to have this available upstream.

[straight-shoota]

This doesn't look like it needs to be part of the static file handler. In fact, it could be useful for caching any HTTP requests.
So maybe this could be implemented in a separate component as a general-purpose caching middleware?

See https://stackoverflow.com/questions/47121991/kemal-caching-responses-with-handler-middleware and https://github.com/jwaldrip/http_cache_handler/blob/master/src/http_cache_handler.cr for inspiration.

[syeopite]

I think any general purpose cache of route responses can quickly balloon in both memory usage and complexity when you take into account query parameters (which can change responses) and headers (eg if the user requested compression, and for what format).

And for this particular use case its just not very efficient. To actually be able to prevent a high-load application from opening millions of file descriptors for example you'll basically have to keep multiple versions of the same file (response) in memory. At minimum:

1. A file response compressed with gzip
2. A file response compressed with deflate
3. An uncompressed file response

And that's not including all the combinations of partial content requests you could receive when a Range header is used. All that will have to be cached to serve the initial purpose.

A general purpose response cache handler can exist (though in my opinion its something that can only be used for very simple responses) but the StaticFileHandler needs some sort of built-in system over something that just caches the resulting responses due to what I mentioned above.

[straight-shoota]

A caching layer that can serve a general purpose doesn't mean it must cache all kinds of requests. You can still limit a separate cache component to serve only a subset of requests (e.g. those served from static files).

[syeopite]

Yeah that's basically what I said as well. A general cache handler can exist but static files would need something specially designed to avoid the issues I mentioned above.

[straight-shoota]

No, I'm saying that a general purpose cache could be able to be configured in a way that the downsides you mentioned do not apply.
Hence, I think it might make more sense to focus on a general purpose cache than a special-case implementation just for the static file handler.